Digital image processing apparatus, tracking method, recording medium for storing computer program for executing the tracking method, and digital image processing apparatus adopting the tracking method

ABSTRACT

A digital image processing apparatus and tracking method are provided to rapidly and accurately track a subject location in video images. The apparatus searches for a target image that is most similar to a reference image, in a current frame image in which each pixel has luminance, and other, data, the reference image being smaller than the current frame image, and includes a similarity calculator for calculating a degree of similarity between the reference image and each of a plurality of matching images that have the same size as the reference image and are portions of the current frame image; and a target image determination unit for determining one of the plurality of matching images as the target image using the degree of similarity obtained by the similarity calculator. The similarity calculator calculates the degree of similarity by applying greater weight to the other data than to the luminance data.

CROSS-REFERENCE TO RELATED PATENT APPLICATION

This application claims the benefit of Korean Patent Application No.10-2009-0022746, filed on Mar. 17, 2009, in the Korean IntellectualProperty Office, the disclosure of which is incorporated herein in itsentirety by reference.

BACKGROUND

The invention relates to a digital image processing apparatus, atracking method, a recording medium for storing a computer program forexecuting the tracking method, and a digital image processing apparatusadopting the tracking method, and more particularly, to a digital imageprocessing apparatus, a tracking method, a recording medium for storinga computer program for executing the tracking method, and a digitalimage processing apparatus adopting the tracking method to rapidly andaccurately track the location of a certain subject in a video image.

In general, a digital image processing apparatus displays a video imagecorresponding to data of a video file stored in a recording medium on adisplay unit. In particular, a digital photographing apparatus (anexample of a digital image processing apparatus) displays a video image(a live-view video image) on a display unit or stores a video file in arecording medium by using data obtained from light incident on animaging device.

The digital photographing apparatus has a tracking function for trackingthe location of a certain subject in a video image. The trackingfunction is performed by determining a portion of a frame image which ismost similar to a reference image (template) as a target image (trackingregion). The reference image is an image including the entire subject ora portion of the subject to be tracked and is smaller than the frameimage.

However, in a conventional digital image processing apparatus having atracking function, an error may occur when determining a portion of eachframe image which is most similar to a reference image as a targetimage, and thus a tracking error may easily occur due to an inaccuratedetermination of the target image.

SUMMARY

An embodiment of the invention provides a digital image processingapparatus, a tracking method, a recording medium for storing a computerprogram for executing the tracking method, and a digital imageprocessing apparatus adopting the tracking method to rapidly andaccurately track the location of a subject in a video image.

According to an embodiment of the invention, there is provided a digitalimage processing apparatus for searching for a target image that is mostsimilar to a reference image, in a current frame image in which eachpixel has luminance data and other data, the reference image beingsmaller than the current frame image, the apparatus including asimilarity calculation unit for calculating a degree of similaritybetween the reference image and each of a plurality of matching imagesthat have the same size as the reference image and is a portion of thecurrent frame image; and a target image determination unit fordetermining one of the plurality of matching images as the target imageby using the degree of similarity that is obtained by the similaritycalculation unit, wherein the similarity calculation unit calculates thedegree of similarity by applying a greater weight to the other data thanto the luminance data.

Each pixel of the current frame image may have luminance data Y, firstchromaticity data C1, and second chromaticity data C2, and, if it isassumed that Y_(R)(i,j), C1 _(R)(i,j), and C2 _(R)(i,j) respectivelyrepresent luminance data, first chromaticity data, and secondchromaticity data of a pixel (i,j) in the reference image, andY_(M)(i,j), C1 _(M)(i,j), and C2 _(M)(i,j) respectively representluminance data, first chromaticity data, and second chromaticity data ofa pixel (i,j) in a matching image, the similarity calculation unit maycalculate a degree of similarity S between the reference image and thematching image by using Equation 1 wherein α<β, α<γ, and α, β, and γ arepositive weights.

S=αΣ|Y _(R)(i,j)−Y_(M)(i,j)|+βΣ|C1_(R)(i,j)−C1_(M)(i,j)|+γΣ|C2_(R)(i,j)−C2_(M)(i,j)|  (1)

Each pixel of the current frame image may have luminance data Y, firstchromaticity data C1, and second chromaticity data C2, and, if it isassumed that Y_(R)(i,j), C1 _(R)(i,j), and C2 _(R)(i,j) respectivelyrepresent luminance data, first chromaticity data, and secondchromaticity data of a pixel (i,j) in the reference image, andY_(M)(i,j), C1 _(M)(i,j), and C2 _(M)(i,j) respectively representluminance data, first chromaticity data, and second chromaticity data ofa pixel (i,j) in a matching image, the similarity calculation unit maycalculate a degree of similarity S between the reference image and thematching image by using Equation 2 wherein α<β, α<γ, and α, β, and γ arepositive weights.

S=αΣ|Y _(R)(i,j)−Y _(M)(i,j)|² +βΣ|C1_(R)(i,j)−C1_(M)(i,j)|²+γΣ|C2_(R)(i,j)−C2_(M)(i,j)|²  (2)

Each pixel of the current frame image may have luminance data Y, firstchromaticity data C1, and second chromaticity data C2, and, if it isassumed that Y_(R)(i,j), C1 _(R)(i,j), and C2 _(R)(i,j) respectivelyrepresent luminance data, first chromaticity data, and secondchromaticity data of a pixel (i,j) in the reference image, andY_(M)(i,j), C1 _(M)(i,j), and C2 _(M)(i,j) respectively representluminance data, first chromaticity data, and second chromaticity data ofa pixel (i,j) in a matching image, and Y_(R)(i,j)-Y_(M)(i,j), C1_(R)(i,j)−C1 _(M)(i,j), and C2 _(R)(i,j)−C2 _(M)(i,j) respectivelyrepresent a luminance value, a first chromaticity value, and a secondchromaticity value, the similarity calculation unit may calculate adegree of similarity S between the reference image and the matchingimage by using Equation 3 wherein α<β, α<γ, and α, β, and γ are positiveweights.

S=α×{luminance value}+β×{first chromaticity value}+γ×{secondchromaticity value}  (3)

β=γ may be satisfied.

The target image determination unit may determine one of the pluralityof matching images which has the lowest degree of similarity with thereference image as the target image.

According to another embodiment of the invention, there is provided atracking method of searching for a target image that is most similar toa reference image, in a current frame image in which each pixel hasluminance data and other data, the reference image being smaller thanthe current frame image, the method including calculating a degree ofsimilarity between the reference image and each of a plurality ofmatching images that have the same size as the reference image and is aportion of the current frame image by applying a greater weight to theother data than to the luminance data.

Each pixel of the current frame image may have luminance data Y, firstchromaticity data C1, and second chromaticity data C2, and, if it isassumed that Y_(R)(i,j), C1 _(R)(i,j), and C2 _(R)(i,j) respectivelyrepresent luminance data, first chromaticity data, and secondchromaticity data of a pixel (i,j) in the reference image, andY_(M)(i,j), C1 _(M)(i,j), and C2 _(M)(i,j) respectively representluminance data, first chromaticity data, and second chromaticity data ofa pixel (i,j) in a matching image, a degree of similarity S between thereference image and the matching image may be calculated by usingEquation 1 wherein α<β, α<γ, and α, β, and γ are positive weights.

S=αΣ|Y _(R)(i,j)−Y_(M)(i,j)|+βΣ|C1_(R)(i,j)−C1_(M)(i,j)|+γΣ|C2_(R)(i,j)−C2_(M)(i,j)|  (1)

Each pixel of the current frame image may have luminance data Y, firstchromaticity data C1, and second chromaticity data C2, and, if it isassumed that Y_(R)(i,j), C1 _(R)(i,j), and C2 _(R)(i,j) respectivelyrepresent luminance data, first chromaticity data, and secondchromaticity data of a pixel (i,j) in the reference image, andY_(M)(i,j), C1 _(M)(i,j), and C2 _(M)(i,j) respectively representluminance data, first chromaticity data, and second chromaticity data ofa pixel (i,j) in a matching image, a degree of similarity S between thereference image and the matching image may be calculated by usingEquation 2 wherein α<β, α<γ, and α, β, and γ are positive weights.

S=αΣ|Y _(R)(i,j)−Y _(M)(i,j)|² +βΣ|C1_(R)(i,j)−C1_(M)(i,j)|²+γΣ|C2_(R)(i,j)−C2_(M)(i,j)|²  (2)

Each pixel of the current frame image may have luminance data Y, firstchromaticity data C1, and second chromaticity data C2, and, if it isassumed that Y_(R)(i,j), C1 _(R)(i,j), and C2 _(R)(i,j) respectivelyrepresent luminance data, first chromaticity data, and secondchromaticity data of a pixel (i,j) in the reference image, andY_(M)(i,j), C1 _(M)(i,j), and C2 _(M)(i,j) respectively representluminance data, first chromaticity data, and second chromaticity data ofa pixel (i,j) in a matching image, and Y_(R)(i,j)-Y_(M)(i,j), C1_(R)(i,j)-C1 _(M)(i,j), and C2 _(R)(i,j)-C2 _(M)(i,j) respectivelyrepresent a luminance value, a first chromaticity value, and a secondchromaticity value, a degree of similarity S between the reference imageand the matching image may be calculated by using Equation 3 whereinα<β, α<γ, and α, β, and γ are positive weights.

S=α×{luminance value}+β×{first chromaticity value}+γ×{secondchromaticity value}  (3)

β=γ may be satisfied.

One of the plurality of matching images which has the lowest degree ofsimilarity with the reference image may be determined as the targetimage.

According to another embodiment of the invention, there is provided acomputer program product, comprising a computer usable medium having acomputer readable program code embodied therein, said computer readableprogram code adapted to be executed to implement the above-describedmethod.

According to another embodiment of the invention, there is provided adigital image processing apparatus adopting the above-described method.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features and advantages will become more apparent bydescribing in detail exemplary embodiments thereof with reference to theattached drawings in which:

FIG. 1 is a block diagram of a digital photographing apparatus as anexample of a digital image processing apparatus according to anembodiment of the invention;

FIG. 2 is a block diagram of a portion of the digital photographingapparatus illustrated in FIG. 1;

FIG. 3 is a pictorial view of a reference image;

FIG. 4 is a pictorial view of a frame image and a search region;

FIG. 5 is a pictorial view of matching images;

FIGS. 6A through 6C are pictorial views for describing a method ofdetermining a search region;

FIG. 7 is a pictorial view of a frame image and a target image;

FIG. 8 is a pictorial view of a reference image; and

FIG. 9 is a flowchart of a tracking method according to an embodiment ofthe invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Hereinafter, the various embodiments of the invention will be describedin detail with reference to the attached drawings.

FIG. 1 is a block diagram of a digital photographing apparatus as anexample of a digital image processing apparatus according to anembodiment of the invention. FIG. 2 is a block diagram of a portion ofthe digital photographing apparatus illustrated in FIG. 1.

A central processing unit (CPU) 100 controls operations of the digitalphotographing apparatus. The digital photographing apparatus alsoincludes a manipulation unit 200 having a key generating an electricalsignal based on a user's input. An electrical signal generated by themanipulation unit 200 is transmitted to the CPU 100 so that the CPU 100may control the digital photographing apparatus in response to theelectrical signal.

In a shooting mode, when an electrical signal based on a user's input isinput to the CPU 100, the CPU 100 analyzes the electrical signal andcontrols a lens driving unit 11, an iris driving unit 21, and an imagingdevice control unit 31, thereby controlling the location of a lens 10,the degree of openness of an iris 20, and the sensitivity of an imagingdevice 30, respectively. The imaging device 30 generates data fromincident light. An analog/digital (A/D) conversion unit 40 convertsanalog data output from the imaging device 30 into digital data. The A/Dconversion unit 40 may not be included according to characteristics ofthe imaging device 30, for example, if the imaging device 30 is adigital device.

The data output from the imaging device 30 may be input to a digitalsignal processing (DSP) unit 50 directly or via a memory 60. The dataoutput from the imaging device 30 may also be input to the CPU 100, ifnecessary. The memory 60 may be read-only memory (ROM) or random accessmemory (RAM). The DSP unit 50 may perform digital signal processing,such as gamma correction, white balance correction, and the like, ifneeded. Also, as will be described later, the DSP unit 50 may include asimilarity calculation unit 52 and a target image determination unit 54in order to efficiently determine a target image. The target image is aportion of a frame image and corresponds to a reference image. Thesimilarity calculation unit 52 and the target image determination unit54 may be separated from the DSP unit 50 or may be included in anothercomponent of the digital photographing apparatus. The functions of thesimilarity calculation unit 52 and the target image determination unit54 will be described later.

The data output from the DSP unit 50 is transmitted to a display controlunit 81 directly or via the memory 60. The display control unit 81controls a display unit 80 to display a video image. The data outputfrom the DSP unit 50 is input to a storing/reading control unit 71directly or via the memory 60. The storing/reading control unit 71stores the data in a storage medium 70 automatically or in response to asignal based on a user's input. The storing/reading control unit 71 mayread data from a file stored in the storage medium 70, and may input thedata to the display control unit 81 via the memory 60 or anothercomponent so that the display unit 80 displays the video image. Thestorage medium 70 may be detachable or non-detachable from the digitalphotographing apparatus or may be partially both (e.g., built-in memorycombined with, e.g., a removable memory card).

The digital photographing apparatus may not include all of the abovecomponents. In some cases, for example, the lens driving unit 11 and theiris driving unit 21 may not be included as long as the digitalphotographing apparatus includes the imaging device 30, the similaritycalculation unit 52, and the target image determination unit 54.Furthermore, the invention is not limited to the digital photographingapparatus illustrated in FIG. 1 and may also be applied to imageprocessing apparatuses such as a personal digital assistant (PDA) and apersonal multimedia player (PMP). The same principal will be applied toother embodiments to be described later. Also, various changes may bemade. For example, if the digital image processing apparatus is not adigital photographing apparatus, the imaging device 30 may not beincluded. The functions of the similarity calculation unit 52 and thetarget image determination unit 54 will now be described with referenceto FIGS. 3 through 5, 6A through 6C, 7, and 8.

FIG. 3 is a pictorial view of a reference image RI, FIG. 4 is aconceptual view of a frame image FI and a search region SR, and FIG. 5is a conceptual view of matching images MI1, MI2, MI3, MI4, MI5, MI6,and MI7.

The reference image RI is an image of a subject to be tracked. The frameimage FI is a frame image of a video image obtained by the digitalphotographing apparatus. The frame image FI may also be a frame image ofa video image of a file stored in a storage medium. The search region SRis a region in the frame image FI that is compared to the referenceimage RI in order to determine a target image. The target image is aportion of the frame image FI and corresponds to the reference image RI.

The matching images MI1, MI2, MI3, MI4, MI5, MI6, and MI7 are arbitraryimages in the reference image RI and have the same size as the referenceimage RI. Although seven matching images (MI1, MI2, MI3, MI4, MI5, MI6,and MI7) are illustrated in FIG. 5, the number of matching images is notfixed to seven and may be differently set according to, for example, thesize of the search region SR. The matching images MI1, MI2, MI3, MI4,MI5, MI6, and MI7 are candidates for the target image and one of thematching images MI1, MI2, MI3, MI4, MI5, MI6, and MI7, which is mostsimilar to the reference image RI, is determined as the target image.For example, in FIG. 5, the matching image MI5 is most similar to thereference image RI and thus the matching image MI5 is determined as thetarget image.

The search region SR may be determined by using various methods. Ingeneral, when the target image is determined in a (t−1)th frame image,and then re-determined in a (t)th frame image, the location of thetarget image in the (t)th frame image is not greatly different from thatin the (t−1)th frame image. Thus, the search region SR in the (t)thframe image may be determined as a region having a size twice or threetimes of the size of the target image in the (t−1)th frame image withreference to the location of the target image in the (t−1)th frameimage, or a region obtained by adding a certain number of pixels to thewidth and the height of the target image in the (t−1)th frame image withreference to the location of the target image in the (t−1)th frameimage.

Various changes may be made and thus the search region SR in the (t)thframe image may be determined by using another method. For example, thesearch region SR may be determined by using a method disclosed in KoreanPatent Application No. 10-2008-0116370 filed by the present applicant onNov. 21, 2008, which has not been yet published. The method disclosed inKorean Patent Application No. 10-2008-0116370 is incorporated byreference and will be briefly described with reference to FIGS. 6Athrough 6C.

FIGS. 6A through 6C are pictorial views for describing the method ofdetermining a search region disclosed in Korean Patent Application No.10-2008-0116370.

Since the location of a target image in a (t)th frame image FI is notgreatly different from that in a (t−1)th frame image, the search regionis determined with reference to the location of the target image in the(t−1)th frame image. Initially, as illustrated in FIG. 6A, withreference to the location of the target image in the (t−1)th frameimage, the location (coordinates) of a first reference region RA1 thathas the same size as a reference image is fixed in a first direction(e.g., the y-axis direction) in the (t)th the frame image FI, a regionin which the first reference region RA1 moves in a second direction(e.g., the x-axis direction) perpendicular to the first direction isdetermined as a first search region SA1, and a portion of the firstsearch region/area SA1, which is most similar to the reference image, isdetermined as a temporary target image. Then, as illustrated in FIG. 6B,with reference to the location of the temporary target image, thelocation (coordinates) of a second reference region RA2 that has thesame size as the temporary target image is fixed in the second direction(e.g., the x-axis direction), a region in which the second referenceregion RA2 moves in the first direction (e.g., the y-axis direction)perpendicular to the second direction is determined as a second searchregion SA2, and a portion of the second search region SA2, which is mostsimilar to the reference image, is determined as an ultimate targetimage. In this case, as illustrated in FIG. 6C, a region including thefirst and second search regions SA1 and SA2 is a search region SA.

A method of determining a target image in the digital photographingapparatus illustrated in FIG. 1 will now be described with reference toFIGS. 7 and 8.

FIG. 7 is a pictorial view of a frame image FI and a target image TI,and FIG. 8 is a pictorial view of a reference image RI.

A portion of the frame image FI, which corresponds to the referenceimage RI, may be accurately tracked and determined as the target imageTI. Although the size of the reference image RI illustrated in FIG. 8 isgreater than that of the target image TI illustrated in FIG. 7 for thesake of convenience, actually they are equal. However, the target imageTI in the frame image FI may or may not completely match the referenceimage RI because a subject included in a tracking region may be changed.

As described above with reference to FIGS. 3 through 5 and 6A through6C, a plurality of matching images may be set in a search region thatmay be determined by using various methods in the frame image FI. Thus,a degree of similarity between each matching image and the referenceimage RI is calculated and a matching image having the highest degree ofsimilarity is determined as the target image TI. For this, in aconventional digital image processing apparatus, the degree ofsimilarity is calculated by using Equation 4.

S=Σ|Y _(R)(i,j)−Y _(M)(i,j)|+Σ|C1_(R)(i,j)−C1_(M)(i,j)−C2_(M)(i,j)|  (4)

Here, it is assumed that each pixel has YCbCr data and Y, Cb (C1), andCr (C2) respectively represent luminance data, first chromaticity data,and second chromaticity data. In more detail, in Equation 4, it isassumed that Y_(R)(i,j), C1 _(R)(i,j), and C2 _(R)(i,j) respectivelyrepresent the luminance data, the first chromaticity data, and thesecond chromaticity data of a pixel (i,j) in the reference image RI, andY_(M)(i,j), C1 _(M)(i,j), and C2 _(M)(i,j) respectively represent theluminance data, the first chromaticity data, and the second chromaticitydata of the pixel (i,j) in a matching image that is a portion of theframe image FI, has the same size as the reference image RI, and iscompared to the reference image RI.

However, in the case of the conventional digital image processingapparatus using Equation 4, since the size of Y is greater than the sizeof C1 or C2 in the YCbCr data of each pixel, the calculated degree ofsimilarity causes a tracking result in which a luminance difference ismore dominant than a chromaticity difference. Accordingly, in variouscases, for example, when the intensity of luminance provided to asubject to be tracked is instantly changed due to a luminance effectcaused by headlights of a car passing by the subject at night, theconventional digital image processing apparatus may inaccuratelydetermine the target image TI and thus may not perform rapid andaccurate tracking.

However, in order to search for the frame image FI, in which each pixelhas luminance data and other data, in order to determine the targetimage TI that is most similar to the reference image RI smaller than theframe image FI, the digital photographing apparatus illustrated in FIG.1 includes the similarity calculation unit 52 for calculating the degreeof similarity between the reference image RI and each matching imagethat is a portion of the frame image FI and has the same size as thereference image RI, and the target image determination unit 54 fordetermining one of the matching images as the target image TI by usingthe degree of similarity, which is obtained by the similaritycalculation unit 52, and the similarity calculation unit 52 calculatesthe degree of similarity by applying a greater weight to the other datathan to the luminance data. As such, unlike the conventional digitalimage processing apparatus, even in various cases, for example, when theintensity of luminance is changed, the target image TI may be accuratelydetermined and thus rapid and accurate tracking may be performed.

In more detail, each pixel of the frame image FI has luminance data Y,first chromaticity data C1, and second chromaticity data C2. If it isassumed that Y_(R)(i,j), C1 _(R)(i,j), and C2 _(R)(i,j) respectivelyrepresent the luminance data, the first chromaticity data, and thesecond chromaticity data of a pixel (i,j) in the reference image RI, andY_(M)(i,j), C1 _(M)(i,j), and C2 _(M)(i,j) respectively represent theluminance data, the first chromaticity data, and the second chromaticitydata of the pixel (i,j) in a matching image, the similarity calculationunit 52 may calculate a degree of similarity S between the referenceimage RI and the matching image by using Equation 1.

S=αΣ|Y _(R)(i,j)−Y_(M)(i,j)|+βΣ|C1_(R)(i,j)−C1_(M)(i,j)|+γΣ|C2_(R)(i,j)−C2_(M)(i,j)|  (1)

Here, α<β, α<γ, and α, β, and γ are positive weights having constantvalues. However, a method of calculating a degree of similarity in thesimilarity calculation unit 52 of the digital image processing apparatusis not limited to Equation 1. For example, Equation 2 may also be used.

S=αΣ|Y _(R)(i,j)−Y_(M)(i,j)|²+βΣ|C1_(R)(i,j)−C1_(M)(i,j)|²+γΣ|C2_(R)(i,j)−C2_(M)(i,j)|²  (2)

In more detail, each pixel of the frame image FI has luminance data Y,first chromaticity data C1, and second chromaticity data C2. If it isassumed that Y_(R)(i,j), C1 _(R)(i,j), and C2 _(R)(i,j) respectivelyrepresent the luminance data, the first chromaticity data, and thesecond chromaticity data of a pixel (i,j) in the reference image RI,Y_(M)(i,j), C1 _(M)(i,j), and C2 _(M)(i,j) respectively represent theluminance data, the first chromaticity data, and the second chromaticitydata of the pixel (i,j) in a matching image, and Y_(R)(i,j)−Y_(M)(i,j),C1 _(R)(i,j)−C1 _(M)(i,j), and C2 _(R)(i,j)-C2 _(M)(i,j) respectivelyrepresent a luminance value, a first chromaticity value, and a secondchromaticity value, the degree of similarity between the reference imageRI and the matching image may be calculated by using Equation 3 whereinα<β, α<γ, and α, β, and γ are positive weights.

S=α×{luminance value}+β×{first chromaticity value}+γ×{secondchromaticity value}  (3)

In Equations 1 through 3, α<β, α<γ, and α, β, and γ are positive weightshaving constant values. Thus, for example, if YCbCr data has a data sizeratio of 4:2:2, α=1 and β=γ=2. In general, the data size of the firstchromaticity data C1 may be equal to the data size of the secondchromaticity data C2 and thus β=γ. If the data size of the firstchromaticity data C1 is different from the data size of the secondchromaticity data C2, β≠γ.

When the similarity calculation unit 52 calculates the degree ofsimilarity between each matching image and the reference image RI asdescribed above, the target image determination unit 54 determines oneof the matching images which has the lowest degree of similarity withthe reference image RI as the target image TI.

Although the digital photographing apparatus is described as arepresentative example of a digital image processing apparatus, theinvention is not limited thereto.

FIG. 9 is a flowchart of a tracking method according to an embodiment ofthe invention.

Referring to FIG. 9, a degree of similarity between a reference imageand each of a plurality of matching images that are portions of acurrent frame image and has the same size as the reference image iscalculated by applying a greater weight to other data than to luminancedata (operation S10). Then, one of the matching images is determined asa target image by using the degree of similarity (operation S20). As agreater weight is applied to the other data than to the luminance datawhen the degree of similarity is calculated (operation S10), unlike aconventional tracking method using Equation 4, even in various cases,for example, when the intensity of luminance is changed, the targetimage may be accurately determined and thus rapid and accurate trackingmay be performed.

The calculating of the degree of similarity (operation S10) will now bedescribed in detail.

If it is assumed that each pixel of the current frame image hasluminance data Y, first chromaticity data C1, and second chromaticitydata C2, Y_(R)(i,j), C1 _(R)(i,j), and C2 _(R)(i,j) respectivelyrepresent the luminance data, the first chromaticity data, and thesecond chromaticity data of a pixel (i,j) in the reference image, andY_(M)(i,j), C1 _(M)(i,j), and C2 _(M)(i,j) respectively represent theluminance data, the first chromaticity data, and the second chromaticitydata of the pixel (i,j) in a matching image, a degree of similarity Sbetween the reference image and the matching image is calculated byusing Equation 1 wherein α<β, α<γ, and α, β, and γ are positive weights.As defined above, and repeated here:

S=αΣ|Y_(R)(i,j)−Y_(M)(i,j)|+βΣ|C1_(R)(i,j)−C1_(M)(i,j)|+γΣ|C2_(R)(i,j)−C2_(M)(i,j)|  (1)

However, a method of calculating a degree of similarity is not limitedto Equation 1. For example, Equation 2 may also be used.

S=αΣ|Y _(R)(i,j)−Y _(M)(i,j)|² +γΣ|C2_(R)(i,j)−C2_(M)(i,j)|²  (2)

In more detail, if it is assumed that each pixel of the current frameimage has luminance data Y, first chromaticity data C1, and secondchromaticity data C2, Y_(R)(i,j), C1 _(R)(i,j), and C2 _(R)(i,j)respectively represent the luminance data, the first chromaticity data,and the second chromaticity data of the pixel (i,j) in the referenceimage, Y_(M)(i,j), C1 _(M)(i,j), and C2 _(M)(i,j) respectively representthe luminance data, the first chromaticity data, and the secondchromaticity data of the pixel (i,j) in a matching image, andY_(R)(i,j)−Y_(M)(i,j), C1 _(R)(i,j)−C1 _(M)(i,j), and C2 _(R)(i,j)−C2_(M)(i,j) respectively represent a luminance value, a first chromaticityvalue, and a second chromaticity value, the degree of similarity Sbetween the reference image and the matching image may be calculated byusing Equation 3 wherein α<β, α<γ, and α, β, and γ are positive weights.

S=α×{luminance value}+β×{first chromaticity value}+γ×{secondchromaticity value}  (3)

In Equations 1 through 3, α<β, α<γ, and α, β, and γ are positive weightshaving constant values. Thus, for example, if YCbCr data has a data sizeratio of 4:2:2,α=1 and β=γ=2. In general, the data size of the firstchromaticity data C1 may be equal to the data size of the secondchromaticity data C2 and thus β=γ. If the data size of the firstchromaticity data C1 is different from the data size of the secondchromaticity data C2, β≠γ.

The determining of the target image (operation S20) may includedetermining one of the matching images, which has the lowest degree ofsimilarity with the reference image, as the target image.

A computer program that can be executed on a processor for executing thetracking method in the digital photographing apparatus may be stored ina recording medium. The recording medium may be, for example, the memory60 or the storage medium 70 illustrated in FIG. 1, or any otherrecording medium such as a magnetic storage medium (e.g., ROM, a floppydisk, or a hard disk) or an optical recording media (e.g., compact disc(CD)-ROM or a digital versatile disc (DVD)).

In addition, the invention is not limited to the digital imageprocessing apparatus described above with reference to FIG. 1. Anydigital image processing apparatus adopting the tracking methodillustrated in FIG. 9 is included in the scope of the invention.

As described above, according to an embodiment of the invention, adigital image processing apparatus, a tracking method, a recordingmedium for storing a computer program for executing the tracking method,and a digital image processing apparatus adopting the tracking method,which may rapidly and accurately track the location of a subject in avideo image, may be realized.

The system or systems described herein may comprise a processor, amemory for storing program data and executing it, a permanent storagesuch as a disk drive, a communications port for handling communicationswith external devices, and user interface devices, including a display,keyboard, mouse, etc. When software modules are involved, these softwaremodules may be stored as program instructions or computer readable codesexecutable on the processor on a computer-readable media such asread-only memory (ROM), random-access memory (RAM), CD-ROMs, magnetictapes, floppy disks, and optical data storage devices. The computerreadable recording medium can also be distributed over network coupledcomputer systems so that the computer readable code is stored andexecuted in a distributed fashion. This media can be read by thecomputer, stored in the memory, and executed by the processor.

All references, including publications, patent applications, andpatents, cited herein are hereby incorporated by reference to the sameextent as if each reference were individually and specifically indicatedto be incorporated by reference and were set forth in its entiretyherein.

For the purposes of promoting an understanding of the principles of theinvention, reference has been made to the preferred embodimentsillustrated in the drawings, and specific language has been used todescribe these embodiments. However, no limitation of the scope of theinvention is intended by this specific language, and the inventionshould be construed to encompass all embodiments that would normallyoccur to one of ordinary skill in the art.

Embodiments of the invention may be described in terms of functionalblock components and various processing steps. Such functional blocksmay be realized by any number of hardware and/or software componentsconfigured to perform the specified functions. For example, embodimentsof the invention may employ various integrated circuit components, e.g.,memory elements, processing elements, logic elements, look-up tables,and the like, which may carry out a variety of functions under thecontrol of one or more microprocessors or other control devices.Similarly, where the elements of the invention are implemented usingsoftware programming or software elements the invention may beimplemented with any programming or scripting language such as C, C++,Java, assembler, or the like, with the various algorithms beingimplemented with any combination of data structures, objects, processes,routines or other programming elements. Functional aspects may beimplemented in algorithms that execute on one or more processors.Furthermore, embodiments of the invention could employ any number ofconventional techniques for electronics configuration, signal processingand/or control, data processing and the like. The words “mechanism” and“element” are used broadly and are not limited to mechanical or physicalembodiments, but can include software routines in conjunction withprocessors, etc.

The particular implementations shown and described herein areillustrative examples of the invention and are not intended to otherwiselimit the scope of the invention in any way. For the sake of brevity,conventional electronics, control systems, software development andother functional aspects of the systems (and components of theindividual operating components of the systems) may not be described indetail. Furthermore, the connecting lines, or connectors shown in thevarious figures presented are intended to represent exemplary functionalrelationships and/or physical or logical couplings between the variouselements. It should be noted that many alternative or additionalfunctional relationships, physical connections or logical connectionsmay be present in a practical device. Moreover, no item or component isessential to the practice of the invention unless the element isspecifically described as “essential” or “critical”.

The use of the terms “a” and “an” and “the” and similar referents in thecontext of describing the invention (especially in the context of thefollowing claims) are to be construed to cover both the singular and theplural. Furthermore, recitation of ranges of values herein are merelyintended to serve as a shorthand method of referring individually toeach separate value falling within the range, unless otherwise indicatedherein, and each separate value is incorporated into the specificationas if it were individually recited herein. Finally, the steps of allmethods described herein can be performed in any suitable order unlessotherwise indicated herein or otherwise clearly contradicted by context.The use of any and all examples, or exemplary language (e.g., “such as”)provided herein, is intended merely to better illuminate the inventionand does not pose a limitation on the scope of the invention unlessotherwise claimed. Numerous modifications and adaptations will bereadily apparent to those skilled in this art without departing from thespirit and scope of the invention.

1. A digital image processing apparatus for searching for a target imagethat is most similar to a reference image, in a current frame image inwhich each pixel has luminance data and other data, the reference imagebeing smaller than the current frame image, the apparatus comprising: asimilarity calculation unit for calculating a degree of similaritybetween the reference image and each of a plurality of matching imagesthat have a same size as the reference image and are portions of thecurrent frame image; and a target image determination unit fordetermining one of the plurality of matching images as the target imageby using the degree of similarity that is obtained by the similaritycalculation unit, wherein the similarity calculation unit calculates thedegree of similarity by applying a greater weight to the other data thanto the luminance data.
 2. The apparatus of claim 1, wherein: each pixelof the current frame image has luminance data Y, first chromaticity dataC1, and second chromaticity data C2; and if it is assumed thatY_(R)(i,j), C1 _(R)(i,j), and C2 _(R)(i,j) respectively representluminance data, first chromaticity data, and second chromaticity data ofa pixel (i,j) in the reference image, and Y_(M)(i,j), C1 _(M)(i,j), andC2 _(M)(i,j) respectively represent luminance data, first chromaticitydata, and second chromaticity data of a pixel (i,j) in a matching image,the similarity calculation unit calculates a degree of similarity Sbetween the reference image and the matching image by using Equation 1S=αΣ|Y _(R)(i,j)−Y_(M)(i,j)|+βΣ|C1_(R)(i,j)−C1_(M)(i,j)+γΣ|C2_(R)(i,j)−C2_(M)(i,j)|  (1)where α<β, α<γ, and α, β, and γ are positive weights
 3. The apparatus ofclaim 2, wherein β=γ.
 4. The apparatus of claim 1, wherein: each pixelof the current frame image has luminance data Y, first chromaticity dataC1, and second chromaticity data C2, and if it is assumed thatY_(R)(i,j), C1 _(R)(i,j), and C2 _(R)(i,j) respectively representluminance data, first chromaticity data, and second chromaticity data ofa pixel (i,j) in the reference image, and Y_(M)(i,j), C1 _(M)(i,j), andC2 _(M)(i,j) respectively represent luminance data, first chromaticitydata, and second chromaticity data of a pixel (i,j) in a matching image,the similarity calculation unit calculates a degree of similarity Sbetween the reference image and the matching image by using Equation 2S=αΣ|Y _(R)(i,j)−Y _(M)(i,j)|² +βΣ|C1_(R)(i,j)−C1_(M)(i,j)|²+γΣ|C2_(R)(i,j)−C2_(M)(i,j)|²  (2) where α<β, α<γ, and α, β, and γ arepositive weights.
 5. The apparatus of claim 4, wherein β=γ.
 6. Theapparatus of claim 1, wherein: each pixel of the current frame image hasluminance data Y, first chromaticity data C1, and second chromaticitydata C2, and if it is assumed that Y_(R)(i,j), C1 _(R)(i,j), and C2_(R)(i,j) respectively represent luminance data, first chromaticitydata, and second chromaticity data of a pixel (i,j) in the referenceimage, and Y_(M)(i,j), C1 _(M)(i,j), and C2 _(M)(i,j) respectivelyrepresent luminance data, first chromaticity data, and secondchromaticity data of a pixel (i,j) in a matching image, andY_(R)(i,j)−Y_(M)(i,j), C1 _(R)(i,j)−C1 _(M)(i,j), and C2 _(R)(i,j)−C2_(M)(i,j) respectively represent a luminance value, a first chromaticityvalue, and a second chromaticity value, the similarity calculation unitcalculates a degree of similarity S between the reference image and thematching image by using Equation 3S=α×{luminance value}+β×{first chromaticity value}+γ×{secondchromaticity value}  (3) where α<β, α<γ, and α, β, and γ are positiveweights.
 7. The apparatus of claim 6, wherein β=γ.
 8. The apparatus ofclaim 1, wherein the target image determination unit determines one ofthe plurality of matching images which has the lowest degree ofsimilarity with the reference image as the target image.
 9. A trackingmethod of searching for a target image that is most similar to areference image, in a current frame image in which each pixel hasluminance data and other data, the reference image being smaller thanthe current frame image, the method comprising: calculating, using aprocessor of a digital image processing apparatus, a degree ofsimilarity between the reference image and each of a plurality ofmatching images that have a same size as the reference image and areportions of the current frame image by applying a greater weight to theother data than to the luminance data.
 10. The method of claim 9,wherein each pixel of the current frame image has luminance data Y,first chromaticity data C1, and second chromaticity data C2, the methodfurther comprising: calculating a degree of similarity S between thereference image and a matching image using Equation 1S=αΣ|Y_(R)(i,j)−Y_(M)(i,j)|+βΣ|C1_(R)(i,j)−C1_(M)(i,j)|+γΣ|C2_(R)(i,j)−C2_(M)(i,j)|  (1),and where α<β, α<γ, and α, β, and γ are positive weights; and whereY_(R)(i,j), C1 _(R)(i,j), and C2 _(R)(i,j) respectively representluminance data, first chromaticity data, and second chromaticity data ofa pixel (i,j) in the reference image, and Y_(M)(i,j), C1 _(M)(i,j), andC2 _(M)(i,j) respectively represent luminance data, first chromaticitydata, and second chromaticity data of a pixel (i,j) in a matching image.11. The method of claim 10, wherein β=γ.
 12. The method of claim 9,wherein each pixel of the current frame image has luminance data Y,first chromaticity data C1, and second chromaticity data C2, the methodfurther comprising: calculating a degree of similarity S between thereference image and a matching image using Equation 2S=αΣ|Y _(R)(i,j)−Y _(M)(i,j)|² +βΣ|C1_(R)(i,j)−C1_(M)(i,j)|²+γΣC2_(R)(i,j)−C2_(M)(i,j)|²   (2) where α<β, α<γ, and α, β, and γ arepositive weights; and where Y_(R)(i,j), C1 _(R)(i,j), and C2 _(R)(i,j)respectively represent luminance data, first chromaticity data, andsecond chromaticity data of a pixel (i,j) in the reference image, andY_(M)(i,j), C1 _(M)(i,j), and C2 _(M)(i,j) respectively representluminance data, first chromaticity data, and second chromaticity data ofthe pixel (i,j) in a matching image.
 13. The method of claim 12, whereinβ=γ.
 14. The method of claim 9, wherein each pixel of the current frameimage has luminance data Y, first chromaticity data C1, and secondchromaticity data C2, the method further comprising: calculating adegree of similarity S between the reference image and a matching imageusing Equation 3S=α×{luminance value}+β×{first chromaticity value}+γ×{secondchromaticity value}  (3) where α<β, α<γ, and α, β, and γ are positiveweights; and where Y_(R)(i,j), C1 _(R)(i,j), and C2 _(R)(i,j)respectively represent luminance data, first chromaticity data, andsecond chromaticity data of a pixel (i,j) in the reference image, andY_(M)(i,j), C1 _(M)(i,j), and C2 _(M)(i,j) respectively representluminance data, first chromaticity data, and second chromaticity data ofpixel (i,j) in a matching image, and Y_(R)(i,j)−Y_(M)(i,j), C1_(R)(i,j)−C1 _(M)(i,j), and C2 _(R)(i,j)−C2 _(M)(i,j) respectivelyrepresent a luminance value, a first chromaticity value, and a secondchromaticity value.
 15. The method of claim 14, wherein β=γ.
 16. Themethod of claim 9, further comprising: determining one of the pluralityof matching images which has the lowest degree of similarity with thereference image to be the target image.
 17. A computer program product,comprising a computer usable medium having a computer readable programcode embodied therein, said computer readable program code adapted to beexecuted to implement a tracking method of searching for a target imagethat is most similar to a reference image, in a current frame image inwhich each pixel has luminance data and other data, the reference imagebeing smaller than the current frame image, the method comprising:calculating, using a processor of a digital image processing apparatus,a degree of similarity between the reference image and each of aplurality of matching images that have a same size as the referenceimage and are portions of the current frame image by applying a greaterweight to the other data than to the luminance data.
 18. A digital imageprocessing apparatus that searches for a target image that is mostsimilar to a reference image, in a current frame image in which eachpixel has luminance data and other data, the reference image beingsmaller than the current frame image, and calculates a degree ofsimilarity between the reference image and each of a plurality ofmatching images that have a same size as the reference image and areportions of the current frame image by applying a greater weight to theother data than to the luminance data.